As part of this process, outputting the alpha coefficients.
## **Wave 1**
## aspfin bfa_mt bfas_ac bfas_ap bfas_ci bfas_co bfas_ea
## alpha 0.8007216 0.7954608 0.88408 0.7814323 0.8559713 0.7748649 0.8882828
## bfas_ee bfas_nv bfas_nv9 bfas_nw bfas_oi bfas_oo
## alpha 0.8547331 0.9113937 0.9010348 0.8781388 0.8287349 0.7899743
## bfi_a bfi_a6 bfi_c bfi_d_scale bfi_e bfi_hp8
## alpha 0.8009146 0.7908462 0.819993 0.7966031 0.8749101 0.6762593
## bfi_n bfi_o bfi_s_scale hrz_col hrz_ind usi
## alpha 0.8582059 0.7898225 0.7969331 0.7126198 0.550577 0.6732525
## vrt_col vrt_ind
## alpha 0.6743606 0.6523363
## **Wave 2**
## aspfin bfa_mt bfas_ac bfas_ap bfas_ci bfas_co
## alpha 0.8252445 0.7878581 0.8829338 0.7785236 0.8492025 0.7839002
## bfas_ea bfas_ee bfas_nv bfas_nv9 bfas_nw bfas_oi
## alpha 0.8761572 0.8590842 0.8972032 0.8854566 0.8698163 0.8153715
## bfas_oo bfi_a bfi_a6 bfi_c bfi_d_scale bfi_e
## alpha 0.7970451 0.7962796 0.7961057 0.8071378 0.7858279 0.8664445
## bfi_hp8 bfi_n bfi_o bfi_s_scale hrz_col hrz_ind
## alpha 0.6848438 0.8587661 0.7966047 0.7848152 0.6932522 0.5938064
## usi vrt_col vrt_ind
## alpha 0.6929115 0.7115297 0.6393136
## **Wave 3**
## aspfin bfa_mt bfas_ac bfas_ap bfas_ci bfas_co
## alpha 0.8338066 0.8046391 0.8862414 0.7768773 0.8492295 0.7774236
## bfas_ea bfas_ee bfas_nv bfas_nv9 bfas_nw bfas_oi
## alpha 0.8819575 0.8616749 0.9007862 0.8903142 0.8817477 0.8376555
## bfas_oo bfi_a bfi_a6 bfi_c bfi_d_scale bfi_e
## alpha 0.8028977 0.803164 0.7982502 0.8213291 0.7909743 0.8728148
## bfi_hp8 bfi_n bfi_o bfi_s_scale hrz_col hrz_ind
## alpha 0.6738761 0.8582405 0.8133423 0.7918976 0.7079314 0.5391312
## usi vrt_col vrt_ind
## alpha 0.6881401 0.6990938 0.6752587
## **Wave 4**
## aspfin bfa_mt bfas_ac bfas_ap bfas_ci bfas_co bfas_ea
## alpha 0.8347909 0.8156071 0.8874039 0.7633623 0.8518579 0.7937478 0.881371
## bfas_ee bfas_nv bfas_nv9 bfas_nw bfas_oi bfas_oo
## alpha 0.8705236 0.9131165 0.9049279 0.8847419 0.8329141 0.8025353
## bfi_a bfi_a6 bfi_c bfi_d_scale bfi_e bfi_hp8
## alpha 0.8117976 0.8028806 0.8218647 0.8056326 0.8749207 0.665913
## bfi_n bfi_o bfi_s_scale hrz_col hrz_ind usi
## alpha 0.8745566 0.8100603 0.7937251 0.7041649 0.5645062 0.6852414
## vrt_col vrt_ind
## alpha 0.7266396 0.6538177
This is an example of the measurement model diagram, using Horizontal Collectivism, for one group demonstrating constraints for the strict measurement invariance model. Paths with the same label are constrained to have the same path weight, and in the strict invariance model, all labeled path weights must be the same for each decade group. The residual covariances are an exception: the constraint imposed is that residual covariance is the same across the same time-lag, for the same indicator (so e.g., cov(aind9, bind9) = cov(bind9, cind9), and cov(aind9, cind9) = cov(bind9, dind9)), but these are not constrained to be the same across group. Since this constraint is imposed in every factorial invariance model tested, it does not affect the fit comparisons.
## hrz_col_W1 ~ 0*1
## hrz_col_W1 ~~ 1*hrz_col_W1
## hrz_col_W1 =~ c(L1, L1, L1, L1)*aind9 + c(L2, L2, L2, L2)*aind10 + c(L3, L3, L3, L3)*aind11 + c(L4, L4, L4, L4)*aind12
## hrz_col_W2 =~ c(L1, L1, L1, L1)*bind9 + c(L2, L2, L2, L2)*bind10 + c(L3, L3, L3, L3)*bind11 + c(L4, L4, L4, L4)*bind12
## hrz_col_W3 =~ c(L1, L1, L1, L1)*cind9 + c(L2, L2, L2, L2)*cind10 + c(L3, L3, L3, L3)*cind11 + c(L4, L4, L4, L4)*cind12
## hrz_col_W4 =~ c(L1, L1, L1, L1)*dind9 + c(L2, L2, L2, L2)*dind10 + c(L3, L3, L3, L3)*dind11 + c(L4, L4, L4, L4)*dind12
##
## #---
##
## aind9 ~ c(int_1, int_1, int_1, int_1)*1
## bind9 ~ c(int_1, int_1, int_1, int_1)*1
## cind9 ~ c(int_1, int_1, int_1, int_1)*1
## dind9 ~ c(int_1, int_1, int_1, int_1)*1
##
## aind10 ~ c(int_2, int_2, int_2, int_2)*1
## bind10 ~ c(int_2, int_2, int_2, int_2)*1
## cind10 ~ c(int_2, int_2, int_2, int_2)*1
## dind10 ~ c(int_2, int_2, int_2, int_2)*1
##
## aind11 ~ c(int_3, int_3, int_3, int_3)*1
## bind11 ~ c(int_3, int_3, int_3, int_3)*1
## cind11 ~ c(int_3, int_3, int_3, int_3)*1
## dind11 ~ c(int_3, int_3, int_3, int_3)*1
##
## aind12 ~ c(int_4, int_4, int_4, int_4)*1
## bind12 ~ c(int_4, int_4, int_4, int_4)*1
## cind12 ~ c(int_4, int_4, int_4, int_4)*1
## dind12 ~ c(int_4, int_4, int_4, int_4)*1
##
## #---
##
## aind9 ~~ c(v_1, v_1, v_1, v_1)*aind9
## bind9 ~~ c(v_1, v_1, v_1, v_1)*bind9
## cind9 ~~ c(v_1, v_1, v_1, v_1)*cind9
## dind9 ~~ c(v_1, v_1, v_1, v_1)*dind9
##
## aind10 ~~ c(v_2, v_2, v_2, v_2)*aind10
## bind10 ~~ c(v_2, v_2, v_2, v_2)*bind10
## cind10 ~~ c(v_2, v_2, v_2, v_2)*cind10
## dind10 ~~ c(v_2, v_2, v_2, v_2)*dind10
##
## aind11 ~~ c(v_3, v_3, v_3, v_3)*aind11
## bind11 ~~ c(v_3, v_3, v_3, v_3)*bind11
## cind11 ~~ c(v_3, v_3, v_3, v_3)*cind11
## dind11 ~~ c(v_3, v_3, v_3, v_3)*dind11
##
## aind12 ~~ c(v_4, v_4, v_4, v_4)*aind12
## bind12 ~~ c(v_4, v_4, v_4, v_4)*bind12
## cind12 ~~ c(v_4, v_4, v_4, v_4)*cind12
## dind12 ~~ c(v_4, v_4, v_4, v_4)*dind12
##
## #---
##
## aind9 ~~ c(cv_11g1, cv_11g2, cv_11g3, cv_11g4)*bind9
## aind9 ~~ c(cv_12g1, cv_12g2, cv_12g3, cv_12g4)*cind9
## aind9 ~~ c(cv_13g1, cv_13g2, cv_13g3, cv_13g4)*dind9
## bind9 ~~ c(cv_11g1, cv_11g2, cv_11g3, cv_11g4)*cind9
## bind9 ~~ c(cv_12g1, cv_12g2, cv_12g3, cv_12g4)*dind9
## cind9 ~~ c(cv_11g1, cv_11g2, cv_11g3, cv_11g4)*dind9
##
## aind10 ~~ c(cv_21g1, cv_21g2, cv_21g3, cv_21g4)*bind10
## aind10 ~~ c(cv_22g1, cv_22g2, cv_22g3, cv_22g4)*cind10
## aind10 ~~ c(cv_23g1, cv_23g2, cv_23g3, cv_23g4)*dind10
## bind10 ~~ c(cv_21g1, cv_21g2, cv_21g3, cv_21g4)*cind10
## bind10 ~~ c(cv_22g1, cv_22g2, cv_22g3, cv_22g4)*dind10
## cind10 ~~ c(cv_21g1, cv_21g2, cv_21g3, cv_21g4)*dind10
##
## aind11 ~~ c(cv_31g1, cv_31g2, cv_31g3, cv_31g4)*bind11
## aind11 ~~ c(cv_32g1, cv_32g2, cv_32g3, cv_32g4)*cind11
## aind11 ~~ c(cv_33g1, cv_33g2, cv_33g3, cv_33g4)*dind11
## bind11 ~~ c(cv_31g1, cv_31g2, cv_31g3, cv_31g4)*cind11
## bind11 ~~ c(cv_32g1, cv_32g2, cv_32g3, cv_32g4)*dind11
## cind11 ~~ c(cv_31g1, cv_31g2, cv_31g3, cv_31g4)*dind11
##
## aind12 ~~ c(cv_41g1, cv_41g2, cv_41g3, cv_41g4)*bind12
## aind12 ~~ c(cv_42g1, cv_42g2, cv_42g3, cv_42g4)*cind12
## aind12 ~~ c(cv_43g1, cv_43g2, cv_43g3, cv_43g4)*dind12
## bind12 ~~ c(cv_41g1, cv_41g2, cv_41g3, cv_41g4)*cind12
## bind12 ~~ c(cv_42g1, cv_42g2, cv_42g3, cv_42g4)*dind12
## cind12 ~~ c(cv_41g1, cv_41g2, cv_41g3, cv_41g4)*dind12
library(semPlot)
semPaths(demo_model_fit, ask = F, include = 1, node.width = .75, layout = 'tree2', rotation = 2, curvature = 9,
intStyle = 'multi', levels = c(1,5,6,7), edge.color = '#bbbbbb', edge.label.color = '#444444',
nCharEdges = 5, edge.label.cex = .5, nCharNodes = 8)
To determine the invariance of measurement over groups, we can examine the \(\chi^2\) test. But this is often an overly strict test with a large sample size. Relative change in AIC and BIC (with lower values, and negative changes, being better) help guide interpretation by incorporating information about the number of parameters (AIC, BIC) and sample size (BIC). Cheung & Rensvold (2002) recomend using \(\Delta\text{CFI}\) because it is not influenced by complexity or sample size, and does not correlate with overall fit measures. In this paper, they write, “A value of \(\Delta\text{CFI}\) smaller than or equal to –0.01 indicates that the null hypothesis of invariance should not be rejected” (p 251). Below, I generally ignore the \(\chi^2\) tests in interpretation but present them for completeness.
Note: The baseline model adds to the strict longitudinal invariance model the additional constraint that covariance among residuals for the same item with the same lag distance must be the same.
| Type | \(\chi^2\) | \(\text{Df}\) | \(\Delta\text{AIC}\) | \(\Delta\text{BIC}\) | \(\text{CFI}\) | \(\Delta\text{CFI}\) | \(\text{MFI}\) | \(\Delta\text{MFI}\) | \(\text{RMSEA}\) | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | unconstrained | 798 | 536 | 0.973 | 0.860 | 0.048 [0.041, 0.054] | ||||
| 2 | long_metric | 864 | 584 | -30.4 | -259.1 | 0.971 | -0.002 | 0.851 | -0.009 | 0.047 [0.040, 0.054] |
| 3 | long_strong | 930 | 632 | -29.3 | -258.0 | 0.969 | -0.002 | 0.842 | -0.009 | 0.047 [0.040, 0.053] |
| 4 | long_strict | 1015 | 692 | -35.4 | -321.2 | 0.966 | -0.003 | 0.830 | -0.012 | 0.046 [0.040, 0.052] |
| 5 | baseline | 1143 | 752 | 8.3 | -277.5 | 0.959 | -0.007 | 0.798 | -0.032 | 0.049 [0.043, 0.055] |
| 6 | metric | 1167 | 767 | -6.2 | -77.7 | 0.958 | -0.001 | 0.794 | -0.004 | 0.049 [0.043, 0.055] |
| 7 | strong | 1206 | 782 | 8.7 | -62.8 | 0.956 | -0.002 | 0.783 | -0.011 | 0.050 [0.044, 0.056] |
| 8 | strict | 1235 | 797 | -0.8 | -72.2 | 0.954 | -0.001 | 0.777 | -0.006 | 0.050 [0.045, 0.056] |
Strong longitudnal invariance. Metric invariance between groups is well supported.
| Type | \(\chi^2\) | \(\text{Df}\) | \(\Delta\text{AIC}\) | \(\Delta\text{BIC}\) | \(\text{CFI}\) | \(\Delta\text{CFI}\) | \(\text{MFI}\) | \(\Delta\text{MFI}\) | \(\text{RMSEA}\) | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | unconstrained | NA [NA, NA] | ||||||||
| 2 | long_metric | 0.729 | 0.036 | 0.068 [0.066, 0.070] | ||||||
| 3 | long_strong | 0.727 | -0.002 | 0.035 | -0.001 | 0.067 [0.066, 0.069] | ||||
| 4 | long_strict | 0.727 | 0.000 | 0.035 | 0.000 | 0.066 [0.065, 0.068] | ||||
| 5 | baseline | 0.722 | -0.005 | 0.033 | -0.002 | 0.066 [0.064, 0.068] | ||||
| 6 | metric | 0.722 | 0.000 | 0.033 | 0.000 | 0.066 [0.064, 0.068] | ||||
| 7 | strong | 0.719 | -0.003 | 0.032 | -0.001 | 0.066 [0.064, 0.068] | ||||
| 8 | strict | 0.719 | -0.001 | 0.032 | -0.000 | 0.066 [0.064, 0.068] |
Possibly problematic. Unconstrained model does not converge. MFI may be too low for accurate model comparison.
| Type | \(\chi^2\) | \(\text{Df}\) | \(\Delta\text{AIC}\) | \(\Delta\text{BIC}\) | \(\text{CFI}\) | \(\Delta\text{CFI}\) | \(\text{MFI}\) | \(\Delta\text{MFI}\) | \(\text{RMSEA}\) | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | unconstrained | 427 | 296 | 0.973 | 0.927 | 0.045 [0.035, 0.054] | ||||
| 2 | long_metric | 480 | 332 | -18.6 | -190.1 | 0.969 | -0.004 | 0.918 | -0.009 | 0.045 [0.036, 0.054] |
| 3 | long_strong | 553 | 368 | 0.4 | -171.1 | 0.962 | -0.008 | 0.899 | -0.019 | 0.048 [0.040, 0.056] |
| 4 | long_strict | 609 | 416 | -40.1 | -268.8 | 0.960 | -0.002 | 0.895 | -0.004 | 0.046 [0.038, 0.054] |
| 5 | baseline | 700 | 464 | -5.1 | -233.8 | 0.951 | -0.009 | 0.873 | -0.022 | 0.048 [0.041, 0.056] |
| 6 | metric | 732 | 476 | 8.4 | -48.8 | 0.947 | -0.004 | 0.863 | -0.010 | 0.050 [0.043, 0.057] |
| 7 | strong | 803 | 488 | 47.1 | -10.0 | 0.935 | -0.012 | 0.834 | -0.029 | 0.055 [0.048, 0.061] |
| 8 | strict | 848 | 500 | 21.2 | -36.0 | 0.928 | -0.007 | 0.818 | -0.016 | 0.057 [0.050, 0.063] |
Longitudinal metric invariance is supported. Group metric invariance is on the recommended cutoff.
| Type | \(\chi^2\) | \(\text{Df}\) | \(\Delta\text{AIC}\) | \(\Delta\text{BIC}\) | \(\text{CFI}\) | \(\Delta\text{CFI}\) | \(\text{MFI}\) | \(\Delta\text{MFI}\) | \(\text{RMSEA}\) | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | unconstrained | 474 | 296 | 0.942 | 0.902 | 0.053 [0.044, 0.061] | ||||
| 2 | long_metric | 507 | 332 | -39.5 | -211.0 | 0.944 | 0.001 | 0.904 | 0.002 | 0.049 [0.041, 0.058] |
| 3 | long_strong | 551 | 368 | -27.3 | -198.8 | 0.941 | -0.003 | 0.900 | -0.005 | 0.048 [0.039, 0.056] |
| 4 | long_strict | 616 | 416 | -31.0 | -259.7 | 0.935 | -0.005 | 0.891 | -0.009 | 0.047 [0.039, 0.055] |
| 5 | baseline | 700 | 464 | -12.5 | -241.1 | 0.924 | -0.011 | 0.873 | -0.018 | 0.048 [0.041, 0.056] |
| 6 | metric | 717 | 476 | -7.0 | -64.2 | 0.922 | -0.002 | 0.870 | -0.003 | 0.048 [0.041, 0.055] |
| 7 | strong | 744 | 488 | 3.5 | -53.6 | 0.917 | -0.005 | 0.862 | -0.008 | 0.049 [0.042, 0.056] |
| 8 | strict | 759 | 500 | -9.3 | -66.5 | 0.916 | -0.001 | 0.861 | -0.001 | 0.049 [0.042, 0.056] |
Invariant.
| Type | \(\chi^2\) | \(\text{Df}\) | \(\Delta\text{AIC}\) | \(\Delta\text{BIC}\) | \(\text{CFI}\) | \(\Delta\text{CFI}\) | \(\text{MFI}\) | \(\Delta\text{MFI}\) | \(\text{RMSEA}\) | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | unconstrained | 22996 | 8408 | 0.599 | 0.000 | 0.090 [0.088, 0.091] | ||||
| 2 | long_metric | 23260 | 8600 | -119.9 | -1034.6 | 0.597 | -0.002 | 0.000 | -0.000 | 0.089 [0.087, 0.090] |
| 3 | long_strong | 23513 | 8792 | -131.4 | -1046.1 | 0.596 | -0.002 | 0.000 | -0.000 | 0.088 [0.087, 0.089] |
| 4 | long_strict | 23793 | 8996 | -128.2 | -1100.0 | 0.594 | -0.002 | 0.000 | -0.000 | 0.087 [0.086, 0.089] |
| 5 | baseline | 24214 | 9200 | 13.3 | -958.5 | 0.588 | -0.006 | 0.000 | -0.000 | 0.087 [0.085, 0.088] |
| 6 | metric | 24287 | 9251 | -29.0 | -271.9 | 0.587 | -0.001 | 0.000 | -0.000 | 0.087 [0.085, 0.088] |
| 7 | strong | 24506 | 9302 | 117.2 | -125.7 | 0.582 | -0.005 | 0.000 | -0.000 | 0.087 [0.086, 0.088] |
| 8 | strict | 24672 | 9353 | 63.7 | -179.3 | 0.579 | -0.003 | 0.000 | -0.000 | 0.087 [0.086, 0.088] |
Possibly problematic.
| Type | \(\chi^2\) | \(\text{Df}\) | \(\Delta\text{AIC}\) | \(\Delta\text{BIC}\) | \(\text{CFI}\) | \(\Delta\text{CFI}\) | \(\text{MFI}\) | \(\Delta\text{MFI}\) | \(\text{RMSEA}\) | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | unconstrained | NA [NA, NA] | ||||||||
| 2 | long_metric | 0.876 | 0.619 | 0.065 [0.061, 0.070] | ||||||
| 3 | long_strong | 0.872 | -0.004 | 0.611 | -0.008 | 0.064 [0.060, 0.069] | ||||
| 4 | long_strict | 0.868 | -0.004 | 0.601 | -0.010 | 0.063 [0.058, 0.067] | ||||
| 5 | baseline | 0.862 | -0.006 | 0.588 | -0.013 | 0.062 [0.058, 0.066] | ||||
| 6 | metric | 0.860 | -0.002 | 0.583 | -0.005 | 0.062 [0.058, 0.066] | ||||
| 7 | strong | 0.853 | -0.008 | 0.566 | -0.017 | 0.063 [0.059, 0.067] | ||||
| 8 | strict | 0.846 | -0.007 | 0.551 | -0.015 | 0.064 [0.060, 0.068] |
The unconstrained model did not converge. Strict longitudinal invariance may be supported. Group metric invariance is supported, but not strong group invariance.
| Type | \(\chi^2\) | \(\text{Df}\) | \(\Delta\text{AIC}\) | \(\Delta\text{BIC}\) | \(\text{CFI}\) | \(\Delta\text{CFI}\) | \(\text{MFI}\) | \(\Delta\text{MFI}\) | \(\text{RMSEA}\) | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | unconstrained | 352 | 296 | 0.988 | 0.968 | 0.029 [0.014, 0.041] | ||||
| 2 | long_metric | 376 | 332 | -47.4 | -218.9 | 0.991 | 0.002 | 0.975 | 0.006 | 0.025 [0.003, 0.037] |
| 3 | long_strong | 413 | 368 | -35.0 | -206.5 | 0.990 | -0.000 | 0.974 | -0.001 | 0.024 [0.000, 0.035] |
| 4 | long_strict | 473 | 416 | -35.7 | -264.3 | 0.988 | -0.003 | 0.967 | -0.007 | 0.025 [0.009, 0.036] |
| 5 | baseline | 568 | 464 | -2.0 | -230.6 | 0.978 | -0.010 | 0.942 | -0.025 | 0.032 [0.022, 0.041] |
| 6 | metric | 601 | 476 | 9.5 | -47.6 | 0.973 | -0.005 | 0.930 | -0.012 | 0.035 [0.025, 0.043] |
| 7 | strong | 660 | 488 | 34.8 | -22.3 | 0.963 | -0.010 | 0.906 | -0.025 | 0.040 [0.032, 0.048] |
| 8 | strict | 683 | 500 | -1.0 | -58.2 | 0.961 | -0.002 | 0.900 | -0.006 | 0.041 [0.033, 0.049] |
Longitudinal strict invariance is well supported. Group metric invariance is not supported.
| Type | \(\chi^2\) | \(\text{Df}\) | \(\Delta\text{AIC}\) | \(\Delta\text{BIC}\) | \(\text{CFI}\) | \(\Delta\text{CFI}\) | \(\text{MFI}\) | \(\Delta\text{MFI}\) | \(\text{RMSEA}\) | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | unconstrained | 343 | 296 | 0.989 | 0.973 | 0.027 [0.009, 0.039] | ||||
| 2 | long_metric | 380 | 332 | -35.2 | -206.7 | 0.989 | -0.000 | 0.973 | -0.000 | 0.026 [0.007, 0.037] |
| 3 | long_strong | 413 | 368 | -38.9 | -210.4 | 0.990 | 0.001 | 0.974 | 0.002 | 0.024 [0.000, 0.035] |
| 4 | long_strict | 468 | 416 | -40.8 | -269.5 | 0.988 | -0.002 | 0.970 | -0.004 | 0.024 [0.006, 0.035] |
| 5 | baseline | 561 | 464 | -3.1 | -231.7 | 0.978 | -0.010 | 0.945 | -0.025 | 0.031 [0.020, 0.040] |
| 6 | metric | 580 | 476 | -5.0 | -62.2 | 0.977 | -0.002 | 0.942 | -0.004 | 0.032 [0.021, 0.041] |
| 7 | strong | 642 | 488 | 38.1 | -19.1 | 0.965 | -0.011 | 0.915 | -0.027 | 0.038 [0.030, 0.046] |
| 8 | strict | 655 | 500 | -10.8 | -68.0 | 0.965 | -0.000 | 0.914 | -0.001 | 0.038 [0.029, 0.046] |
Longitudinal strict invaraince is well supported. Group metric invariance is well supported.